Magnitude Estimation and the Non-Linearity of Acceptability Judgments

The term experimental syntax – the use of psycholinguistic methodologies for the collection of acceptability judgments – can cover any number of designs, tasks, and statistical analyses (Cowart 1997, Schütze 1996). Over the past decade, one task in particular, the magnitude estimation task, has received significant attention for its alleged ability to provide more accurate data, almost to the point of becoming a ‘gold standard’ among judgment collection techniques (Bard et al. 1996, Keller 2000, and Keller 2003). While magnitude estimation has been a staple task of psychophysics for over 50 years (Stevens 1957), it has only become a part of linguistic methodology thanks to the demonstration by Bard, Robertson, and Sorace (1996) that it could be profitably adapted for the collection of acceptability judgments. In the ensuing decade, magnitude estimation has been applied to a number of areas of syntactic research with exciting results (e.g., Featherston 2005a, 2005b, Sorace and Keller 2005); however, since that seminal paper there has been relatively little research into the task itself. For example, given that magnitude estimation was originally developed to measure the perception of physical stimuli, there may be certain assumptions built into the magnitude estimation task that may not be compatible with the perception of linguistic acceptability. This paper investigates one such assumption: that participants are able to use the modulus to estimate acceptability along a linear scale. As we shall see shortly, there are two manners in which the magnitude estimation task could be used to estimate acceptability along a linear scale, yet the four experiments presented in this paper suggest that neither manner is actually adopted by the participants. These results indicate that linguistic magnitude estimation differs significantly from psychophysical magnitude estimation, which may necessitate a reconsideration of one of the methodological advantages that have been offered for the widespread adoption of linguistic magnitude estimation by syntacticians: that the use of a modulus stimulus leads to more accurate measures of acceptability. Before beginning the investigation of the linguistic magnitude estimation task, it seems worthwhile to give a brief overview of the magnitude estimation task itself. The magnitude estimation task was originally developed to investigate humans’ perception of physical stimuli. For instance, if the brightness of a light source is doubled, is it the case that we perceive the light as twice as bright as the original? While a priori you may be inclined to answer ‘Yes, of course,’ psychophysicists in the middle of 20 century used the magnitude estimation task to determine that, in fact, you would perceive the light as only 1.4 times as bright. The task itself was straightforward. Participants were presented with a single example of the stimulus, for example a light source, and told that its magnitude (in this case brightness) is 100 units. They were then presented with other examples of the stimulus, and asked to estimate the magnitude of the new stimuli based on the original. If they perceived the second light as twice as bright, they would report a brightness of 200; if they perceived the second light as half as bright, they would report a brightness of 50. The reference stimulus (called the modulus) is kept constant throughout the experiment so that every experimental item is estimated using the same reference. In this way, the experimenters could compare participants’ reported perceptions with the actual physical measurements of the stimulus to determine the nature of human perception for various stimuli (brightness, volume, heat, length, and hundreds of others). As it turns